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A project in Thailand to develop domain-referenced 
English language tests for use with Thai secondary and college 
students is described. The first phase consisted of developing 
standardized tests for listening, speaking, reading, and writing 
based on existing syllabuses, determining the common levels of 
English proficiency in those domains, and analyzing the communi -iv^ 
competence factors in English use of the Thai students. Subjects were 
697 lower secondary, 525 upper secondary, and 493 university 
students. The second phase involved examining the consistency of 
domain specifications and setting domain-content standard scores for 
the tests. The process of test construction and validation are 
outlined, and results for each phase are reported. Recommendations 
for further research in this area include study of the rontritJUtion 
of individual language skills to the development of overall language 
proficiency, with the objective of finding effective remedial 
methods, and similar test and standard development for all levels of 
Thai education. Recommendations for practical application in English 
teaching and learning include use of a communicative context for 
language instruction and early emphasis on oral/aural skills 
gradually leading to later emphasis on reading and writing. A 55-item 
bibliography and test objectives, specifications, and analyses are 
appended. (KSE) 
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fBTRODUCTlOM TO DOMAlt-RBFSREMCED TESTfMG 

Doma i n-ref erenced testing dates back to 1966 when 
Wells Hively of the University of Minnesota started the 
MINNEMAS7 Project aiming at the development of domain- 
referenced tests for science and mathematics. Publications on 
this subject coming out in 1974 and 1975 were influential 
especially ir Minnesota to the widespread use of this new trend 
in testing* Domain-referenced testing expanded on the concept 
of criterion-referenced testing which measures students' 
mastery of the set objectives by building items on the 
incorporated domain description! content limits, critt^rion of 
distractor description, criterion of distractor domains, format 
and direction, and specimen items* Sen^^ion and Rafaehl ( 1974) 
delineated the steps of domain-referenced test construction as 
consisting of: 

!• developing the domain on the results of the 
analyses of content s/ sk i 1 1 s , teaching and learning procedures, 
objectives and 1 earner- cent ered goals which are clear and 
easily understood, and 

2t developing domain-referenced test itoms by 

sampling of testees and content domains, and setting suitable 
test techniques relevant to the teaching- 1 earning situations. 

The results of the tests are to be analyzed and 
reported to all parties concerned, i.e., teachers, students. 
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parentsietc. The report of the student*s profile willf in turn* 
serve the purposes of remedial teaching and both short-term and 
long-^term learning improvement. Information yielded by domain- 
referenced tests is greater than that yielded fay criterion 

2 

referenced tests because it is directly linked to the domains 
of teaching and learning, thus, readily lending itself to 
remedial teaching and learning improvement. In other words, 
domain-referenced testing is a system developed to measure 
learning mastery which goes beyond criterion-referenced testing^ 
by not merely focusing on the criteria of achievement set by 
the behavioral objectives, but pinpointing also on the content 
/skill and the context of evaluation and the environment of 
learning and acquisition as a domain incorporates the areas 
covered by the objectives, the content of learning, the skills 
or sub-skill levels, for instance speaking skill at the 
criticism level, reading skill at the comprehension and 
interpretative levels. Student learning/testing activities, 
such as taking note^, listening to lectures, etc., and the 
learning/testing media, such as video--audio tapes, newspapers, 

OHP,etc. are includ9d in the domain-description. A domain 
must have a definite area with a clear borderline separating it 
from other domains. 

The amplified objectives on which a doma i n-^r ef erencad 
test specification is based are expanded from behavioral 
objectives which have been criticized as nnt being objective 
nor complex enough for measuring human learning. Specificity 
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is one of the necessary dimensions of amplified objectives* 
Roid and Haladyna ( 1982s 297) mentioned the effort to improve 
behavioral objectives by Popham of the University of California 
lOX (Item-Objective Exchang'e)# Popham's amplified objectives 
cons i sts of 5 parts as f o 1 lows t 

1# Description of the traits to be measured or the 

3 

main objectives of testiniJi 

2# Specific objectives which have been transformed 
from the main objectives! 

3. Descriptions of item forms of both the stimuli 
and the responses; in other words, behaviors/performance to be 
tested* types and techniques of testinji the test itemsi format 
and techniques of testees^s responses are specif iedi 

4. Cond i t ion and cr Iter la of correct responses, 
5» Specimen test items* 

Thorndike (1982: 2) described the process of 
developing domain-referenced test items as consisting of 5 
steps as follows: 

1* Specify and limit the domaint 

2t Spec i f y the test techni ques or the format of the 

St imu 1 i ^ 

3» Specify the technique or mode of responsesf 
4. Specify the amount of items unbiasedly sampled 
from the universe of content, 

5i Specify the criteria of mastery of the domain* 

The descriptions of domains-referenced testing" and 
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ampliflad objectives above confirm their behavioral roots. 
With the advent of i nd i v i dua 1 i zed « learner-centered approach of 
the early 1970'S| the domain-referenced item-form shell 
developed by Hivelyi et. al.(Alkin« 1973{ 25) incorporated the 
trans format i on ru 1 e by whi oh items can be subst i tuted by 
paralleled onest and the instructions to be given to individual 
students whose behaviors are to be observed and recorded* 
Lists of stimuli — verfaalf pictorialf realiat etc»---to be 

4 

presented to the testees form a component of an item shell. 

Purves, et. al.<i984) specified the system of giving 
instructions for 9ssay writing in their international study on 
domain-referenced essay writing to include 15 dimensions^ i#e«f 
i nstruct i ons « stimuli « cognitive demand* g'oalt role* audiencof 
conoentf genre* stylei advanced preparation of inf ormat i on * 
lengtht writing patternt time* drafting and criterion. Purves* 
domain specification has proven useful for essay writing as it 
reminds the students of the details and complexities of the 
writing tasks as well as serving as a tool for evaluating an essay. 
ComwanicBhive Appromch to Dommin-rafBirBnced T&sting 
Test ing commun i cat i ve 1 anguage competence is based on 
soc i o-cul tur a 1 and pragu:atic theories of language learning. 
Language contentsi encoding skills and subskills — unitary or 
i nt egrat i ve--are meshed together in the dynamics of 
communication with the following communicative parametersi 
function* modality* channel of communi cat i on i statusi role* 
style* topic and communicative situations. An item shell of a 
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communicative dowa i n-ref erenced test may include discrete-point 
measuring ling'uistfc competencet macro/ ^1 oba 1 items measuring 
soc io 1 in^ui st i c competence! s ingl e/uni tary and integrated- sk i 1 1 
items meaeuring strategic, pragmatic and discourse competrence# 
Testing methods may be either or both selection (multiple 
choice I true/f a lee i matching, sequenc ing, etc . ) and supply 
(cloze, completion, short-answer, letter-writing, essay, etc»)» 

Using B, J. Carroll's (1980) parameters for 
communicative test construction in developing a domain- 
referenced test the following parameters are included in the 

5 

test blueprint: 

1 • Context I time and locat ion of language 

interaction, e.g., in the classroom, at the train-station, in 
the restaurant, etc. 

2. Content Key: main idea or topic of language 
interaction, e,g*, language in bargaining, in business 
transaction, in signing a treaty, in lecturing about danger of 
neuclear wars, in advertising, etc. 

3. Function and Discourse; language for work or 
personal social transsctionp elaborated/restricted code, 
referent i a 1 /d i rect i ve/poet i c/phat i c f unct i on or met a 1 anguage, 
descriptive/persuasive/expl snat or y d i scour ee . 

4. Modality! language interactions via the oral mode 
(listening, speaking? or the visual mode (reading, writing), 

5. Genre J methods of employing language skills, e.g., 
writing a personal letter, writing an essay, taking notes, 
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writing a bele^rsnif talking on the telephone! reading a 
newspaper, etc# 

6« Rolet interaction between lang'uasfd transmitters 
and receiverai e.sf.f teacher-student t father-soTM fr iend-f riendt 
younger brother-elder sister, etc. 

7« Statuss Boc i o-economi c statuses or professional 
areas governing the style and register of language usSf e*g«, 
ecclesiastical language^ court language, vulgar languaget 
language for specific purposes* such as English for airline 
hostesses and stewards and stewardesses, etc. 

8« Pre- suppos i t iont anticipation of language 

transmissions, e.g., expectancy of the incoming message through 

6 

some media or c f ace-^-to-^f ace interact ioni familiarity with the 
language user or transmitter will facilitate communication, for 
example reading a friend's smeared letter with some words or 
phrases missing, or listening to one*s mother in a noisy market 
placet Here communication is not hampered on account of the 
receivers correct anticipation of the incoming message. 

9» Mood or Attitudes! the mood and attitudes of 
language users can determine the content and style of language 
use, e.g., humorous, rude, sincere, sarcastic, negative, 
positive, etc. 

10. Formal ityi the level of language use, e.g*, 
formal, colloquial, slang, academic, standard, sub-standard, etc. 

The aforementioned framework for doma in^ref er enced 
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testing within ^he communicative language teachinsf/l earning 
approach is congTruent with the Ensflish language teaching 
/learning practice currently endorsed in various sectors of the 
Thai educational systems where English is viewed as the 
language of international communication and where meanings of 
the message are the heart of the matter* 

TBB RSSSAHCB 

The research on domain-referenced tests of Eng'lish 
use was conducted on a grant given by the Thai government and 
was administratively supported by the Chu la 1 ongkorn University 
Language Institute and the Testing Bureau of the Ministry of 
Education. The research has two phases. Phase I 
Project was entitled Reearch for the Deve lopment of 

7 

Engl i sh Doma in- Referenced Tests for Use with Tha i 
Students i n Var i ous Leve 1 s > The objectives of Phase I 
Project were toi 

!• Develop standardized domain-referenced test-- 
items from the determined domains of English use via the sound 
modality, i.e., listening and speaking and the graphic modality, 
i.e.i reading and writing, as specified in the English 
syllabuses for the lower secondary, upper secondary and 
tertiary levels of education, 

2. Determine the common levels of English 

proficiency in the specified aomains of students in the three 
levels of education as stated in objective U 1, and to 
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3 . Study , ana lyze and dotermi ne the factors of 
communicative competence in usin^ English of Thai students in 
the lower secondary and tertiary levels of education. 

The mul t i -staje strat i f i ed random samp 1 in? method was 
emp 1 oycd to obta i n the samp 1 e g^roups of 697 1 ower secondary 
students t 525 upper secondary i and 493 Chul a lon^korn 
Uni vers ity students. 

The Rasch Model and the sensitivity and facility 
indices were used for test item analyses and revision. Domain- 
referenced methods of test analyses were used to establish the 
tests' coefficients of reliability and validity* F--tests and t- 
tests were used to establish test cr i ter i on^re lated validity. 

Parametric statistics used to discover students' 
levels of English proficiency in the specified domains of 
English use were arithmetic meant standard deviation! 
coefficient of var i at i onC CV ) i kurtosis and skewness. Students' 

8 

proficiency levels were rated on a 5-band scales from very weak 
requiring extensive remedial worki to very Sood not 
requiring any remedial work. 

To study the factors of communicative competencet 
Exploratory Principal Component Factor Analysis with Varimax 
Rotation and Pearson Product Moment Correlations were employed^ 

The research project in Phase II entitled 
Exam i net i on of the Cons i stency of Doma i n Spec i f 1 cat i ons for Engl i sh 
Language Jest yn the Lower and Upper Secondary Leve 1 s of Educat i on 
aimed at s 
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1« Examining the consistency .>f domain 

specif i cat iors of the dona in-i^ef erenced tests of Snflish use* and 

2* Sett ing the doma in- content standard scores for 
the doina in-ri f erenced tests of English use constructed for the 
res€^8rch* (Details in Appendix IV) 

The sample groups consisted of 563 lower secondary 
and 469 upper secondary students selected by the stratified 
random sampling method* 

Test reliability indices were established using the 
Cronbach Alpha Nehod« The Rasch Model was employed to examine 
the test items' difficulty indices and their congruence with 
the ICC or the item characteristic curve* To prove the 
consistency of domain specifications used to develop three 
forms of doma in-ref enced tests for each level totalling nine 
testSi the F-test method was used on the arithmetic means of 
item difficulty of each form* With the F ratio of 0.00 and the 
probability of 0.99f the consistency of the domain 
specifications for the three parallel forms — Form Af Form Bt 

9 

and Form C was confirmed. 

The percentile method was used to set the test domain- 
content standard scores* (See Appendix IV) 

The research in both of its phases is a language 
testing research in its full sense by beingt 

1. a research to develop three sets of domain-^ 
referenced tests of English use, three f^rms for each sett from 
the English curricula of the lower and upper levels of 
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secondary of education and frow the English syllabuses of the 
Chulalon^korn University Lanffuars Institute for students in 
first, second and third years ftt Chula lonykorn Universityi the 
test iteins were processed through the various steps of test 
analyses for the purpose of test standardization and validation» 

2. a research to find out about th . nature and 
specific characteristics of communicative proficiency in the 
English language by the use of the constructed and standardized 
domain-referenced tests of En'^Hsh uset 

3. a research v.sinjf the domain-referenced tests of 
English use as a tool for studying the levels of English 
ability of Thai students in three levels of education. 

LAtBUAGS TSSTite RSSSARCJB IM TBB FIRST &SMSE 

To develop, standardize, and validate the domain- 
referenced tfcsts of English use for the three levels of 
education — lower secondary, upper secondary, and tertiary — the 
following steps were tahenj 

Step One 

From the laarning objectives specified in the English 

10 

syllebuses amplified objectives were developed. 

(See Appendix 1 for a specimen set of amplified objectives) 
Step Two 

From the amplified objectives domain-referenced teat 
tables of specification were developed to sample the English 
learning domains set in the syllsbuBee and encapsulated in the 



ERIC 



12 



12 

ampl i f ied object i ve8« 

(See Appendix II for a specimen domain-referenced 
test table of specification) 

St&p TJhTBB 

The mult itrait-mult imethod assesement scheme was used 
in test developmentt From the domain-referenced test tables of 
specification the multimethod approach to test construction 
conaiuting of both selection-type ItemSf ie» « multiple choicet 
matching^v and item seQuencing, and supply-type itemSf ie.f 
completion of statements! diagrams and charts* short-answers* 
partial dictationi full dictation, summarization, paragraph 
writing and essays. The multitrait assessment consisted of 
mensuring both lanffuasT^ componants, ie.i grammar* vocabulsry, 
and phonology which are discrete-point items, and language ski list 
unitary skills of listenins^, speakinff, reading and writing and 
integrative skills of listening-writing, listening-reading- 
writing, and reading-writing. All the items were maximal 
performance tests except for the items measuring student 
speaking skill which employed a self-assessment rating scheme. 

For the lowt.* secondary level 225 items, 75 items per 
form, were constructed. Tor the upper secondary level also 
225 i tems , 75 items per form , were deve 1 oped ^ Fo^ the tert iary 

1 1 

level within the context of Chula longkorn University, 210 items, 
70 i tems per form, were constructed . The tests , therefore , 
cons isted of 3 parallel forms for each 1 eve 1 of educat i on . 
Each form was based on the domain-referenced test table of 
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specification developed from the amplified objectives for each 
1 eve 1 1 

StBp Four 

For test standardization the following methods of 
test analysis were performed on the students* obtained raw soonest 

1. The classical model of item analysis, ie,, Chung 
Teh Fan ( 1952) 27% upper-lower method was used to find out item 
difficulty and discrimination indices which were used as a 
basis for item revision. The Whitney & Sabers ( 1970f 1971) 
method was used on students scores from the mult iple^point items, 

ie*, essays, short-answers, partial dictation, full dictation, 
and summarization for the same purpose* 

2. The second pre-tests were conducted using the 
revised test forms on the sampled subjects of the first pre-- 
tests consisting of 42 lower-secondary, 41 upper-secondary, and 
47 Chula longkorn University students. 

3. The doma in-^ref erenced model of item analysis was 
used to est ab 1 i sh test i t em sens i t i v i ty i nd i ces and f ac i 1 i ty 
values using the pre-post test data* Swaminathan, Hambleton 
and Algina's (In Pitiyanuwat, 1986) methods of test analysis 
were used to find the reliability and validity indices of the 
tests* Item-objective congruence was also carried out to 
established the test content validity. 

12 

<See Appendix III for the formulae of test analyses) 

4t Test administration of the main sample groups for 
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the research consisting of 1 , 26C lower secondary* 994 upper 
secondary and 493 tertiary < Chul a 1 ongkorr. University) -students 
using the nine standardized and validated doma i n-ref erunced 
tests of English use was ;-:arriad out. The data were analyzed 
using the Rasch Model to find item-fit statistics which 
indicated that over 95% of the itams fit the ICC. 

Step FivB 

To validate the tests the procrdures emp'oyed 

consisted of the foUowingi 

1. As stated in « 3 of Step Four the domain-referenced 
methods of test analysis were used to establish the tests' 
reliability and validity indices. 

2. The validity established through the item- 
objective congruence method was content validity. 

3. The teats' criterion-related validty was 
established through the pre-post test method mentioned in «3 
of Step Four. Besides the pre-post test method of test 
analysis, the tests ' cr i ter i on-re 1 ated validity was also 
established using t-tests for secondmry students divided into 

2 groups on the basis of school type — small-medium sized, and 
large-very large sized, and F-tests for university students 
divided into 3 groups on the basis of their year of study-- 
first-year, second-yerr, and third-year. The t-values and 
the F-ratios which were statistically significant at .05 and .01 
levels confirmed the tests' criterion-related validity, 

13 
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4, The tests' construct validity was established 
using Pearson Product Moment Correlation method. Correlational 
matrices anion? the various parts of the tests using selection 
and supply item types and measuring different traits via 
unitary skill mode and integrative skill mode were analyzed. 
The correlations were labelled as follows? 

C - Monotra i t-^Heteromethod 

H = Heterotra I t-Heteromethod 

MM = Monotra it -Mopomet hod 

V ^ Total score with part score 

M - Heterotra i t'-Monomethod 
Correlations labelled C should be greater than those labelled M 
and MM whichf in turn^ should hs greater than correlations 
labelled MM if the tests are convergent ly va 1 i d • Besidesi 
C and MM correlations should fa© greater than H if the tests are 
discriminant ly val id (Campbell anu Fiske, 1967). The V 
correlations shoulif also be greater than the MM to prove the 
tests* discriminant validity. 

The correlational matrices for the three sets of 
domain-referenced tests exhibited these patterns of 
correlations indicating the tests ' construct validity of the 
convergent and d i scr i m i nant types . 

Sfcep Six 

Doma i n--cont ent standard scores of the three tests were 
computed using the percentile methoci. Groups of items were 
identified from 5 to 95 percentiles. The grouping of the 
scores from each test exhibited similar patterns indicating the 
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consistency of item difficulty and student ability. 

14 

(See Appendix IV for Domain-Content Standard Scores) 
Stop S&ven 

The general i zabi 1 ity of the domain-referenced tables 
of specification was proven by F-tests of the item difficulty 
values. The F-ratio of 0.00 and the probability of 0.99 on all 
the tests proved that the ICC curves of the three parallel 
tests for each level did not statistically significantly 
deviate from the standard ICC. 

(See Appendix V for the F-tests of the domain- 
referenced tests' ICC *ith the standard ICC) 
LAtGUAGE TEST I MO RSSEARCE IM THE SSCOMD SS9SE 

The research intended to find out about the nature 
and specific characteristics of communicative proficiency In 
English by using the constructed, standardized and validated 
domain-referenced tests of English use. The ressarch 
procedures consisted oft 

Step One 

The analyses using Pearson Product Moment Correlation 
method revealed that language components ♦ i o . , phonology, 
vocabulary andgrammar, weremoderat e 1 y and significantly 
c-jrrrelated with language modalities, ie., oral/aural and 
graphic/visual, and that the shared variances with the oral/aural 
or sound modality were 19X» 35S and Z4% for the lower secondary, 
upper secondary, and tertiary levels respectively. The variances 
shared with the visual modality were 22X, 24%, and 15% respectively. 
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Step Two 

Exploratory principal component factor analyses usin^ 
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varimax rotation were performed on the data obtained through 
the mult itra it-mult Imethod assessment scheme. For the lower 
secondary (students aged 14-15) four points emergedt 

1. At this level the sound modality was finely 
differentiated into 3 factors: Factor One --listening, word- 
stress, and speaking (SSQ 1.458), Factor Two— i ntegrat i ve 
skills of 1 ist,ening-readinff-wr it ing and sound discrimination (SSQ 
1.193) and Factor Three — integrative skills of listening and 
writing (SSQ 1.129). Thus, the shared variances of sound 
modality with total communicative proficiency were 11.2%, 9. OX 
and 8.5X totalling 28.7%, Factor Four was visual modality 
consisting of writing, reading, vocabulary, reading-writing, 
grammar and speaking (measured indirectly via reading). 

2. Vocabulary and grammar were grouped with the 
visual modality indicating that the test mainly measured 
this factor at this level of education. 

3. The visual modality factor shared common variance 
with the total variance of communicatively proficiency by 25%. 

4. The four factors could together explain domain- 
referenced communicative proficiency by 53. 7X 

Factor analysis performed on test snore data of upper- 
secondary students( students aged 17-18) indicated that: 

^* this level of education, communicative 
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proficiency in English consisted of iwo factors. Factor One 
was sound modality consisting of listening, word-stress, and 
speaking assessed through a self-ratin? method. Factor Two 
was visual modality consisting of vocabulary and speaking 
measured indirectly via reading. 

16 

2. while vocabulary was grouped with visual 
variables, grammar was grouped with sound variables. Again the 
indirect test of speaking via reading was grouped under the 
visual modality factor. 

For university level (students aged 17/18-20/21) the 
findintjs indicated that; 

1. domain-referenced communicative proficiency was 
differentiated into three groups of factors with one sound 
modality factor and two visual modality factors. Factor One 
of the visual modality consisted of indirect speaking measured 
via reading, grammar, reading, and vocabulary, Factor Two of 
the visual modality consisted of integrative skills of reading- 
writing and unitary skill of reading. Factor Throe was sound 
modality consisting of sound discrimination, word-stress, 
speaking measured via self-rating, integrative skills of 
listening and writing, and listening-reading-writing. 

2. visual modality factors combined could explain 
the variance of communicative proficiency by 20. 7% while sound 
modality could explain it by 31.5%. 

3. similar to the findings for the lower secondary 
level, grammar and vocabulary were grouped Mth visual modality. 
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4. similar to the findings for both levels of 
Bfi^condary eduoationf the findings for this level confirmed the 
r/rouping of indirect speaking measured via reading with visual 
modality* (See Appendix VI for factor analysis results) 

Conclusions 

From the af orement i oned f inu i ngs for the three 

17 

levels of eduoationf the following conclusions about the nature 
and specific chai^acter i st i cs of communicative proficiency 
could be drawni 

It At the lower intermediate level of proficiency or 
the lower secondary level of eduoationf sound lnod(K^Hty factors 
were most finely differentiated. Converaelyi at the upper 
intermediate level or tertiary level of education the visual 
modality factors k ere most finely differentiated* At the 
intermediate level of proficiency or upper secondary level of 
education each modality had equally only one factor* 

The findings might be the results of the nature or 
process of English acquisition of Thai students as well as thr. 
emphasis differently placed on each modality* At the lower 
intermediate level i oral-aural modes of communication are 
stressed most whereas at the upper-- intermed i ate level i visual 
modality especially the reading skills receive most emphasis. 

2. Language componi?nt variables of 30und« vocabulary 
and graTfimar wore never onc^ grouped together into a 
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differentiated factoPt They were always subsumed under the 
skill f actors--sound discrimination and word-stress with aural 
/oral skills and grammar and vocabulary sosietrimes together 
under the visual nodal ity factor for lower secondary and 
tertiary levels« arn sometimes separately — vocabulary with 
visual modality and grammar with sound modality for upper 
secondary level* 

3# The sound factors across the three levels of 
education could explain the variance of domain-referenced 

18 

communicative proficiency by 28. 7X, 36r9X and 31.5% respectively, 
while the visual modality factors could explain it by 25.0%^ 
9* IX and 20.7% respectively, or together at the ratio of K8ll. 

This illustrated the significant role of sound variables in 
student acquisition of communicative proficiency as they 
appeared to have more common variance with that of total 
comwun i cat i ve prof i c i ency . 

LA96LfA0E TESTIM0 RSSBARCS it THE T'^.RD SSESB 

This research also aimed at discovering the levels of 
stuient Enl ! eh communicat i ve prof i ciency in the spec i f led 
domains in the three levels of education. Parametric 
statistical analyses were performed to find the mean, 
standard deviation, kurtosis, skewness and coefficient of 
variation (CV> of tho tests. The 5 criterion levels were used 
to i dent i f y of student lev^? 1 s of communi cat i ve proficiency--- 

1 - very poor • need max i ma 1 remed i a 1 work 



21 



- M 21 

2 = raiher poor, need remedial work 

3 = average* still need remedial work 

4 = rather good« need some remedial work 

5 = very good, do not need any remedial work 

The findings indicated that for the lower secondary 
students their unitary skills of reading, listening, speaking 
were in general average while their integrated skills were 
quite weak. The CV indicated a high degree of score variation 
( 2-7). 

For the upper secondary students their speaking 

19 

skills were average while their other unitary and Integrated 
skills ranged from rather poor to average. The CV indicated 
an averagw degree of score variation (2-5). 

For university students, they were average to good in 
speaking, average in reading, rather poor to average in the 
integrated skills. The CV indicated an average degree of score 

variation ( 2-5) . 

Across the levels, Thai students ranged from levels 2- 
4 in their unitary skills while their integrated skills ranged 
from levels 2-3. The CV indicated that lower secondary 
students had the greatest degree of score variation (2-7) while 
the upper secondary and university groups' had average degree 
2-5 and 1-4 respectively. (See Appendix VII for students' levels 
of communicative proficiency in English) 
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RSCOMMSMDAnOtS 

The research on domain-referenced testing In both 
phases led to th9 fol lowing recommendat i ons i 

For Farthor R&saarch 
1. Research ir.to the degrees of contribution that 
language components have on acquisition of the unitary and 
integrated language skills should be conducted as well as 
research to find the most effective remedial and instructional 
processes for different levels of proficiency. 

2. Amplified objectivesi domain-referenced test 
tables of specification and domain-referenced test items should 
be developed, standardized and validated for every sub-level of 
primary, secondary, and tertiary education in Thailand. 

20 

For Practicai Appt ications in Sng I i ah Teaching and L&arning 

1. Language components — phonology, grammar, 
vocabulary--should not be presented or practised without the 
context of communication as they were never once grouped into 
a separate factor. 

2. At the beginning to lower intermediate levels or 
with students aged 7 to 14 the sound modality should get more 
emphasis than the visual one if language acquisition is to 
occur naturally, 

3. At the intermediate levels or with students aged 
^° '^'^ * joth modalities may receive equal amount of 

presentation and practice, 

4. At the upper intermediate to advanced levels or 
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with studenis agedl7/18 bo20/21 the unitary skill or integrated 
skills of reading and writing should fae emphasized most # 

5. As the sound mo da 1 i ty factors shared greater 
variance with communicative proficiency (at the ratio of 1#BM)» 
media and materials for listoningi speakingi listening- 
speak ingf 1 i stening-wr it ingf and 1 i steni ng^-resd i ng-wr i t i rg 
skills should be provided for both classrooom instruction and 
self-access learning for individual students in all levels of 
educat ion. 
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APPEtDlX I 

Amptiri&d Object i yes for English Core Cours&i E 6 /Si 6 It 

(Upper Seconder jr Lerel) 

\. After listening to the economic, political, and sports news, 
the student is able to answer four-choice multiple choice 
question about the main ideas and details of the news* 
2* After listening to the news, the student is able to make an 
oral summary of the news. 

3» While listening to descriptions of people, animals, objects 
and places, the student is able to ask for more information 
about the objects of the description. 

4» While listening to narrations about incidents such as an 
accident, the student is able to tai^ e notes of what happened 
and able to sequence the incidents. 

5# After listening to songs and poems, the student is able to 
express his/her evaluative comments on what he/she has heard. 
6# When someone makes a wrong statement about something, e.g., 
a personal relationship, the student is able to correct it. 

7. The student is able to ask for details about people, 
objects, places, time and activities, and is able to 
answer Wh quest i ons accord ingly. 

8. The student is able to make an appropriate oral invitation 
to -'arious groups of people on various occasions. 
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9t The student is able to speak about, his deterininat i on to do 
something in an appropriate context or situatioHt et^.f when 
asked about his/her future rjlan for study# 

10. When a claysm^^e speaks about something in which the 

student is not interested! he/she is able to express his disinterestt 
!!• After reading a letter or a note giving directions to a 
particular placet the student is able to draw a map of the place* 
I2i After reading about a person*s daily routine, the student 
IS able to complete a time-table about that person^s activitiest 
13» After reading captionsf the student is able to match the 
captions with the appropriate pictures. 

14. After reading an assigned short story or novel f the 
sbudent is able to relate it to others* 

15. After reading a cong'ratul atory letter or cardf the student 
is able to write a thank-you reply. 

16. After listening to a telephone messagep the student is 
able to take down the message* 

17. After reading a short story, an article or a novel* the 
student is able to summarize it. 

18. After reading a short story or a novel i the student is 
able to extend it from his imagination. 

19. After reading a "Want** ad, the student Is able to write s 
1 ett er of app 1 i cat i on . 

20. After receiving good or bad news* the student is able to 
write a letter of congratulation or sympathy to the person concerned. 
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APPS9DIX II 

SpBciwBn Domain-ReferBnced TBSt Tmblo of Spac if i cm t ion 



Ob J. 


Langustfe 

P unei^ i on 


Node 


Genre 


Channel 


Style 


Sole 


Status 


Topic 


1 


informs- 
t ion 


List-- 
Speak 


News 


Radio/ 
TV 


Forma 1 


Stu* to 
Peers & 
Teachers 


Equa 1 


Econt 
Pol it • 
Sports 


5 


Express 
Opinion 


Speak/ 
Writ. 


Liter 
ary 


Faoe to 

Face/ 

Radio 


Forma 1 


Student 
to Peers 
&Teacher 


Equa 1 


Point 
of view 


17 


Summar i ze 


Read/ 
Writ. 


Short 
story 

ve 1 

i 


Book/ 
Magaz ine 


Forma 1 


Student 
to Peers 
aTeacher 


Equa 1 


Adven- 
ture 



Test lien Type and Test Technique 
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Item 1 SaloctioTi type — Four-choice Multiple Choice 

Lenfi.h of News — 50-100 words 
Item 5 Supply type — Integrative skills — Listening/Speaking 

and Listening/Writing 

Lenjth of Input (song and poem) 150-200 words 
Length of Output ( Student expression) 100-150 words 
Item lY Supply type — Integrative skills — Reading/Writing 

Length of Input (Short story and article ) 1000-2000 words 



(Novel) 10000-15000 words 
Length of Output (Student summaries) Foir short story 
nove 1 --500-600 words 

For novel — 1000-1500 words 



Marking Scheme 

Item 1 1 for correct 

o for incorrect 

Item 5 On a Rating Scale of 7 from I'no competence, 
2=minimal competence, 3=threshold competence 
4=Bome communicative competence 
S'^quite competent 

6=good-very good communicative competence 
7=near educated native communicative competence 
Item 17 Saipe as Item 5 
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APPSBDIX III 
Formulme for itmm Analjaas 

1. Whiiney & Sabers Multiple-point Item Analysis 
Facility Value (FV> = S + S - (N,) X . 

^ M 1 Twin 

N (X 

T mm H * Kwlrk) 

S = total scores of the upper ^roup 
S = total scores of the lower group 
N = total number of testees used in the analysis 

T 

X ^hiffhest score obtained 

tmm 9€ 

X =1 owest score obta i ned 

D i scr i m i nat i on I ndex ( D ) 
D = S - S 

/////// / ^///v^////^ 
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N (X - X ) 



S fS .X ,X syiDboHze the same values as in the 

formula For FV 

= number of tesLees in the upper tfroup 
Sourcei Whitney ?' Siifaers, 1970» 1971 In Scannell & Tracvt 
1975: 221-224* 

2, Domain-^Ref ©renced Test Item Analysis 

Fac i Hty Val ue ( FV) = Proportion of testees 
passing the each item after instruction 



Sensitivity Index<S^)= R post^ - R pre^ 



R post ^ = Number of testees passing item^ after 
i nst ruct i on 

R pre^ = Nuinber of testees passing the item (jtem^) 

T = Total number of testees 
The values of rang^e between -1 and +1. Good 

items should have the values close to 
Sources Pitiysniiwat, 1986i 79-80. 

3» Index of I tem-^Ofa ject i ve Congruence (Content Validity) 
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1 = ( k - 1 )s - S 

i e o o i 



2N (k - 1) 

1 = Index of item objective congruence for item and 

to t 

Object i ve 

K = Number of objectives to be measured by the test 
N = Number of spec ialist raters 
S - Total scores of all raters for objective 
S Total scores of all raters for all 

object i ves except ob ject i ve 

o 

Sourcei Wibulsri» 1983. 



4. Swaminathan, Hambleton 8f Algina Coefficient of 
Rel iabi 1 ity 

K = P - P 



1 - P 

m 

K - Coefficient of Reliability 

P - Observed Proportion (Proportion of 
testees passing both pre and post tests 4^ proportion of 
testees failing both pre and post tests) 

P = Expected Proportion E (Proportion of 
testees passing both tests Proportion of testees passing 

the pre-test but failing the post test) x (Proportion of 
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iesteas passing bobh tests + Proportion of testees failing^ 
the pre test but passing the post tost) -fr 
(Proportion of testees failing the pre test but passing the post test) 



(Proportion of testees failing both tests) x (Proportion of 
testees passing the pre test but failing the post test -I- 
Proportion of testees failing both tests)] 

Examples Criterion of Passing = 80% 



Pre 
Post 



Pass 



Fai 1 



Total Pass 



Pass 
Fai 1 
Total 



.66 
. 04 
.70 



. 14 
. 16 
. 30 



.80 
.20 
I. 00 



P = , 66 + . 16 = . 82 

P = (.70) (.80) + (.30) + (.20) = .62 

m 

K = .82-. 62 



1-. 62 
= .53 

Domain-Referenced Validity Coefficient 
V = a + c 



3.9 
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a b -f c + d 
V ^ Validiiy Coefficient 

a = Number of people passing the criterion after 
instruct ion 

b = Number of people passing the criterion before 
instruct i on 

c =^ Number of people not passing the criterion before 
instruct i on 

d = Number of people not passing the criterion after 
i nst ruct i on 

Val idity coefficient ranges from 0 - 1 • A 
value close to I indicates test val idity. 
Sources Pitiyanuwat, 19868 79--81 



APPEMDiX tV 
The Tost Domain-Content StsdarcT Scores 

Symbols used in the table 
S = Speaking 
L = Listening 
R - Reading 

ES^Essay or paragraph writing 
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Dl=Dictation ( L i steninff-Wr i t in? ) 
RW= Raadins-Wpiting 
LW=Listeninff-Wr it in? 

LR=Listeninff-ReadiTiff-Writ in? (Partial Dictation) 

L = Lower Level of Secondary Education 
U = Upper Level of Secondary Education 
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The Table reveals that for lower secondary students, the 
items grouped into high, middle and low percent i les measured 
the same skills by 40%. 65%, and 07 % respectively for Forms A, B 
and C averaging 57% of the items measuring thf same skills for 
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the three formst 

For th© upper level of secondary education* the test 
items in the three forms measured the same skills by 49% on the 
average* 

Students in both levels of secondary education could 
do items measuring reading and reiiding bastf second were items 
measuring their speaking skills via a self-rating methodt The 
items they could do least were items measuring their 
integrative skills of 1 i stening-wr it ingi listening-reading- 
wring, read ing-*wr i t ing and essay/paragraph writing* 



APPSMDIX ¥ 

F-t0stB of the Domafn-RBforencBd Tests' ICC ( I torn ChmractBristic 
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Curre> with th& Standard ICC 



Lower Socondarj LbkbI 



Form 


X 


SD 


N 


A 


0.02 


1 . 49 


42 


B 


0.04 


1 .24 


42 


C 


0.04 


1 .30 


39 




SS 


DF 


MS 


Bet* 








Groups 


. 10 


2 


.00 



Within 

Group 211.94 120 
Total 211.95 122 



1.81 



.00 0.99 



Upper Secondar/ Levwt 



Form 


X 


SD 




N 


A 


0.06 


0.91 




35 


B 


-0.02 


1.28 




36 


C 


0. 10 


1.05 




36 


SS 


CP 


MS 


F 


P 


.23 


2 


0. 11 


0.10 


0.99 



117. 44 
117, 67 



104 
106 



1.20 
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Abbrev S at i ons 



APPBWDiX VI 
Factor Anmiyaia Smautta 
FV = Factor for Visual Modalities — Reading, 
Writing, Road i ng-Wr i t inf , Writing, 

indirect Speaking Measured via Reading 
FS = Factor for Sound Modalities — Listening, 

Listening-Reading-Writing, Li stening-Wr iting 
SD = Sound Discrimination, ST=Word Stress 
VOC = Knowledge of vocabulary, G = Use of Grammar 
IS(R) = Indirect Speaking Measured via Reading 



Lower Smcondarjr 

(N« 679) 

FV I FS I FS 1 I FS 1 1 I 

W L L-R-W L-W 

R 



ST 

VOC S 

R-W 

f SCR) 

G 
SSQ 

2.54 1.458 



; D 



1.193 1.128 



Upper Secondarr 



(N 


525) 




(N«493> 




FS I 


FV I 


FSI 


FVI 


FVI I 


L 


Voc 


SD 


ISCR) 


R-W 


ST 


R 


ST 


G 


W 


G 


IS(R) 


S 


R 




S 


R-W 


L-W 


VOC 




L-W 


W 


L'=R-W 






L-R-W 










SSQ 




SSQ 







3.256 



2.447 



Tart i arj 



2.339 1.974 1.426 



APPBMDIX VIl 

Students* Levets of English Proficiencj 
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Snglish Proficimncy tover Smcondmrj Uppmr Sacondary ToTtimry 

Level CV Level CV Level CV 



Lang^uag^e Components 3 1 2-3 2 3 1 

Sound Modalities 2-3 13 2 3 2 

Visual Modal itisB 2-3 2 2-3 2 2-3 2 

CoinponentsaModal i- 2-3 1-7 2-3 2-5 3 1-4 
tias Conibinad 
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